Kaleido Logo

CRL Workshop Application

Cost Readiness Level Assessment System

CRL Workshop Materials

Workshop Introduction

Title: Systems Thinking in Cost Estimation: A Readiness Framework in Action

Duration: 75 minutes

Format: Presentation (20 min), Case Study Introduction (10 min), Interactive Exercises (30 min), Results & Future Directions (15 min)

Workshop Objectives
  • Understand Cost Readiness Levels (CRL) concepts and practical applications
  • Apply CRL frameworks to assess estimation uncertainty in various project contexts
  • Evaluate cost maturity across different work packages and project elements
  • Identify targeted improvement strategies for enhancing cost confidence
Content Plan
1. Workshop Introduction

Setting the stage for understanding CRL concepts and practical applications through a real-world case study.

2. Semiconductor Facility Case Study Overview

Introduction to the SemiTech Inc. project and its estimation challenges.

3. Interactive Exercises
  • Exercise 1: CRL Index at Sub-Project Level
  • Exercise 2: CRL Criteria Schema
  • Exercise 3: CRL by Work / Trade Package
4. Results & Future Directions

Analysis of exercise outcomes, practical implementation strategies, and evolution of the CRL framework.

Semiconductor Facility Case Study Overview

Project Background

Client: SemiTech Inc.

Project: NextGen Semiconductor Manufacturing Facility

Project Description:

SemiTech Inc. is investing $2.85 billion in a state-of-the-art semiconductor manufacturing facility designed to produce next-generation chips for Quantum Computing applications. With a Ground Internal Floor Area of 150,000m², it will be the largest factory of its kind in the world. The investment is 50% from the US tax payer and 50% private equity, hence the government accountability and strict commercial regulations surrounding the project.

Project Purpose:

  • Expand production capacity for advanced semiconductor chips
  • Implement cutting-edge manufacturing processes
  • Reduce production costs through automation and efficiency
  • Position the country competitively in the global high-performance computing market race

Current Project Status:

The project is currently approaching final budget authorization. The building design has reached an advanced stage (approx. 70% complete), while the semiconductor processing equipment design remains in critical developmental stages (TRL 5-6). This disparity in design maturity creates significant challenges for accurate cost estimation and risk management. For example, the special design is in flux as the process design is preliminary.

Commercial Structure

Project Delivery Model: Integrated Project Delivery (IPD) with risk shared with the client at this early stage

Key Project Partners:

  1. Ernie & Wise - Prime Contractor/Managing Consultancy
    • Responsible for overall project management and integration
    • Holds the master contract with SemiTech Inc.
    • Manages commercial arrangements with subcontractors
    • Leading the cost estimation and risk management process
  2. Faker Twix - Design Partner
    • Responsible for facility architectural and engineering design
    • Cleanroom design specialists
    • Building systems integration (HVAC, electrical, utilities)
    • Well-established track record with similar facilities
  3. Nott McDonalds - Construction Partner
    • Responsible for building construction execution
    • Cleanroom construction specialists
    • Mechanical and electrical systems installation
    • Has completed three similar facilities in the past five years
  4. ACME Corporation - Manufacturing Process and Equipment Partner
    • Specialized semiconductor equipment design and integration
    • Novel lithography systems development
    • Process workflow optimization
    • First major project implementing their pioneered 1nm process technology, first of its kind

Contract Structure:

The project is currently planned to be procured through a single turnkey Engineering, Procurement and Construction (EPC) Cost Plus contract with incentives for meeting cost and schedule targets. This contractual approach has been selected due to the complexity and scale of the project, despite differing maturity levels across project components, noting the risk in the investment sat with investors and the government.

Estimation Challenge

The estimation team at Ernie & Wise has built a comprehensive bottom-up detailed estimate for the building and infrastructure components, with some top-down assessments for the process equipment. Monte Carlo Simulation was performed showing:

  • Base cost: $2,850.1 million
  • P50 value: $3,700 million
  • Recommended contingency: $849.9 million (P50 - base)

According to traditional classification systems, the estimate would be classified as Class 2, with an expected accuracy range of -5% to -15% on the low side, and +5% to +20% on the high side. However, the actual results from the probabilistic risk analysis show a range of -8.2% to +68.5%, indicating the classification system may not be capturing the true uncertainty in the project, particularly for the less mature semiconductor process equipment.

This disparity has created tension with senior management, who are pressuring the estimation team to revise their estimate to fit the expected accuracy range of a Class 2 estimate due to affordability concerns. As an estimator/cost engineer, you would like to communicate the readiness of the project estimate given the state of the project's maturity. The estimation team is exploring Cost Readiness Levels as a complementary framework to better communicate the true state of estimate uncertainty.

Exercise 1: CRL Index at Sub-Project Level

Workshop Task

Using the proposed CRL index framework and the information provided in the Basis of Estimate, assess the Cost Readiness Level for each major sub-project (Construction of Factory vs. Manufacturing Process & Equipment). Identify specific characteristics from the CRL definitions that justify your assessment.

Cost Readiness Level (CRL) Framework

The CRL framework provides a standardised approach to evaluate cost estimate maturity:

  • CRL 1-3: Conceptual/Early development - High uncertainty
  • CRL 4-6: Developing maturity - Moderate uncertainty
  • CRL 7-9: Advanced maturity - Lower uncertainty
Basis of Estimate
Building and Infrastructure:

The building and infrastructure components comprise 39.3% of the total project cost ($1,120.0M) but contribute only 37.5% to the overall variance. The estimate is based on the following:

  • Detailed civil and structural engineering designs with 90-95% completion
  • Quantities extracted from 3D BIM models that have been independently verified
  • Multiple contractor and vendor quotations for major elements
  • Detailed site investigations including geotechnical surveys and soil analysis
  • Historical cost data from three similar semiconductor facilities built in the last five years
  • Building services at varying levels of design maturity (30-85% complete)
  • MEP services sized based on preliminary process equipment requirements

The substructure and structural elements have high design maturity with validated quantities and firm price quotes. The architectural works are well advanced in design but have some dependencies on equipment layouts. Building services show the least maturity as they are highly dependent on finalized process equipment specifications.

Manufacturing Process & Equipment:

The manufacturing process and equipment elements account for 60.7% of the total project cost ($1,730.3M) and contribute 62.5% of the overall variance. The estimate is based on:

  • Preliminary process flow diagrams and equipment layouts (40-60% complete)
  • Budgetary quotes from equipment vendors with numerous qualification items
  • Limited historical data for some specialized 1nm process technology systems
  • Parametric cost models and scaling factors from previous generation technology
  • High-level material requirements with significant uncertainty in quantities
  • Benchmarking against less advanced facilities with significant factors applied
  • Preliminary process utility requirements with significant contingency factors

The standard process systems have moderate design maturity with reasonable budgetary pricing. The novel lithography and process control systems have lower design maturity and significant pricing uncertainty. The integration and automation systems are at a conceptual level with high uncertainty in both scope and pricing.

CRL Index Definitions
CRL 1-3: Conceptual Estimate
Level Description
CRL 1 Rough Order of Magnitude (ROM). CRL 1 Initial cost concepts based primarily on expert judgment and preliminary concept assumptions. They often use parametric or rule-of-thumb metrics from similar projects, gross analogies, or very basic calculations. Supporting documentary evidence is minimal, and the estimate includes broad uncertainty factors.
CRL 2 Broad cost concepts with improved clarity around variables & uncertainties through basic estimates. Cost estimates have seen an iterative refinement, using more defined analogies, with early documentation & consistency measures. They may employ early parametric models with improved clarity in assumptions and scope definition, but still have significant uncertainties.
CRL 3 Structured cost elements with quantitative and qualitative analysis. These early estimates feature improved scope definition, allowing for structuring into logical cost elements, and the application of semi-structured estimating methodologies. Increased use of organizational tools like work breakdown structures and early assessment of uncertainties begins to emerge.
CRL 4-6: Preliminary Estimate
Level Description
CRL 4 Semi-detailed cost estimates with documented framework & assumptions. These intermediate estimates start to incorporate detailed estimates for the largest cost driver elements. They feature increased documentation of estimation methodology, scope boundaries, and assumptions. Early identification of uncertainties and risk begins to complement the base estimate development.
CRL 5 Detailed cost estimates for significant elements & probabilistic ranges for other elements. These estimates feature a hybrid approach, with detailed cost estimation for major cost elements, and justified factors or parametric models for less significant elements. Medium-fidelity models may be supplemented with expert interviews, and basic Monte Carlo simulation might be applied to preliminary risk assessments.
CRL 6 Independently verified cost estimates with benchmark validation. These estimates could feature calibrated parametric models and/or detailed cost estimates with a broad degree of fidelity. The integration of multiple estimation approaches begins to provide greater confidence in the results.
CRL 7-9: Detailed Estimate
Level Description
CRL 7 Fully validated cost estimates based on detailed engineering data and/or preliminary vendor quotes. These detailed estimates exhibit a high degree of fidelity with a 'golden thread' auditable trail. They are typically peer-reviewed and/or independently verified and validated, supported by a robust Basis of Estimate, analogous or parametric models, and/or previous phase estimates demonstrating progression in detail and fidelity.
CRL 8 Cost estimates based on firm contractual quotes with residual uncertainty quantified. These estimates represent contracted offers post-negotiation with risk quantified by contract type. They show evidence of continuation from previous phase work and performance, and/or are informed by historical trend analysis or previous performance data.
CRL 9 Reconciled costs based on actual expenditures and comprehensive post-project analysis. These represent fully validated, audited actual costs that have been reconciled and are usable as historical data for future estimates. They represent the highest level of readiness and provide a foundation for continuous improvement with data used to bolster future project assessment.

Exercise 2: CRL Schema Application

Workshop Task

Apply the CRL criteria schema to evaluate each of the five assessment dimensions (Estimate Method, Quality, Uncertainty & Risk, Benchmarking, Contract Type Alignment) for the two major project components. Determine which dimensions are driving the CRL limitations and recommend targeted improvements.

Five CRL Criteria
  • Method: The appropriateness and sophistication of estimation techniques
  • Data Quality: The quality and reliability of input data and assumptions
  • Uncertainty & Risk: The rigour of uncertainty analysis and risk quantification
  • Benchmarking: The extent of comparative analysis with similar projects
  • Contract Type Alignment: The alignment of estimate with procurement strategy
Basis of Estimate
Estimation Methods:
  • Building Elements: Primarily detailed unit cost estimation with quantities from engineering drawings; some parametric estimates for MEP systems
  • Process Equipment: Combination of equipment vendor budgetary quotes, factored parametric estimating, and benchmarking against similar (but less advanced) installations
Data Quality Factors:
  • Building structural design is well advanced with 3D BIM models (70-95% complete)
  • MEP systems specifications defined but undergoing optimisation
  • Equipment data based on preliminary specifications for novel technology
  • Limited historical data exists for 1nm process technology components
  • Wafer processing systems are first-of-kind with minimal reference projects
Uncertainty Assessment Approach:
  • Three-point estimates developed for all major cost elements
  • Monte Carlo simulation applied with triangular distributions
  • Correlation factors applied based on expert judgement
  • Identified specific risk events with probability/impact matrices
  • Sensitivity analysis performed on key cost drivers
Benchmarking Activities:
  • Building costs benchmarked against three similar facilities (2-8 years old)
  • Cleanroom construction costs indexed from previous projects
  • Standard equipment costs compared to previous generation systems
  • Novel equipment extrapolated from R&D prototypes with limited benchmarks
  • Published industry metrics used for high-level validation
Contract Strategy Considerations:
  • Building structure and substructure ready for lump-sum contracts
  • MEP systems suitable for target-cost arrangements with shared risk/reward
  • Standard process equipment appropriate for fixed-price with limited qualifications
  • Novel equipment requiring cost-plus or time & materials arrangements
  • Integration systems needing progressive elaboration contracts
CRL Schema Criteria Definitions
Criterion CRL 1 CRL 2 CRL 3 CRL 4 CRL 5 CRL 6 CRL 7 CRL 8 CRL 9
Method Expert judgment with minimal supporting analysis Parametric models with limited analogous adjustments Structured cost models with significant uncertainty Factored parametric estimates or system analogies Mixed approach combining parametric and detailed estimates Detailed estimates with benchmark validation Fully validated estimates based on detailed engineering Estimates based on firm contractual quotes Reconciled costs based on actual expenditures
Data Quality Single data point or unverified industry averages Limited historical data, minimal validation Some historical or analogous data with notable gaps Moderate historical data with validation Reliable data for major cost elements High-quality data with minimal gaps Comprehensive data set from reliable sources Actual quotes and firm proposals Complete actual cost data
Uncertainty & Risk Qualitative assessment with no formal risk identification General risk categories identified without quantification High-level risk register with limited analysis Initial triangular distributions based on expert judgment Systematic triangular distributions with min/max validation Refined triangular distributions with correlation factors Narrow triangular distributions with stakeholder validation Minimal triangular distribution range for remaining variables No distributions required – actual costs recorded
Benchmarking No benchmarking or reference projects Minimal reference to broadly similar projects Industry benchmarks with significant adjustments Benchmark comparisons with documented adjustments Detailed comparison against similar projects Close analogies with minor adjustments Multiple benchmarks with statistical analysis Direct benchmark mapping with minimal variance Becomes a benchmark for future projects
Contract Type Alignment Unsuitable for any contractual commitment Suitable only for order-of-magnitude budgeting Appropriate for early concept evaluation only Suitable for preliminary budget authorization Appropriate for budget establishment with contingency Suitable for cost-reimbursable contracts with caps Appropriate for target cost arrangements Suitable for fixed-price or lump-sum contracts Used for contract closeout and historical database

Exercise 3: CRL Assessment by Work/Trade Package

Workshop Task

Using the detailed BoQ provided and the Basis of Estimate below, evaluate how CRLs vary across different work and trade packages. Identify optimal contract strategies for each package based on their CRL assessment.

International Cost Management Standard (ICMS) Groups

The International Cost Management Standard (ICMS) provides a standardised way to organise and classify construction costs across different markets. For this exercise, identify which construction packages have higher CRL maturity and which have significant uncertainty.

Project Cost Summary
ICMS Cost code Description Total ($M) % of CC
0 TOTAL Construction Cost (CC) 2850.1 100.0%
Building and Infrastructure
2.02 Substructure 112.8 4.0%
2.03 Structure 72.6 2.6%
2.04 Architectural works │ Non-structural works 71.1 2.5%
2.05 Building Services & Utilities 863.3 30.3%
2.12 Production & Process Equipment 1730.3 60.7%
2.12.010 Etching equipment 352.6 12.4%
2.12.020 Lithography systems 493.9 17.3%
2.12.022 Wafer-processing systems 647.6 22.7%
2.12.030 Metrology & process-control systems 236.2 8.3%
Basis of Estimate - Qualitative Assessment

Each work package has a different level of design maturity, cost certainty, and readiness for procurement. Below is a qualitative assessment of the key work packages:

2.02 Substructure:

Nott McDonalds have completed detailed design for the substructure with 95% design documents. Three validated contractor quotations have been received, with minimal qualification items. Quantities have been extracted from BIM models and have been independently verified. Geotechnical reports have been completed with extensive bore sampling, and foundation design has been optimised. The team is ready to award contracts with fixed pricing for most elements.

2.03 Structure:

The structural design is 90% complete with detailed engineering drawings and specifications. Material quantities have been extracted from the structural model and independently verified. Steel fabricators and concrete suppliers have provided firm quotations with small contingencies for market fluctuations in raw materials. Seismic design has been validated by third-party engineering consultants. The package is ready for final tender.

2.04 Architectural Works:

Architectural designs are approximately 85% complete. The cleanroom envelope design has undergone two design iterations but final vendor selection is pending. External cladding systems have been specified with detailed drawings, and three vendor quotes have been received. Finishes schedules are developed but subject to final coordination with process equipment layout. Most elements have reasonable budgetary prices, with some items still at conceptual pricing.

2.05 Services and Equipment:

MEP designs are at varying stages (30-85% complete). HVAC system sizing is still being optimised based on evolving heat loads from process equipment. Electrical load calculations are based on preliminary equipment data with significant contingency. Major equipment has budgetary quotations, while distribution systems are estimated parametrically. Controls integration has significant design gaps and limited pricing information.

4.01 Wafer Processing Systems:

Standard processing equipment has firm vendor proposals with specified performance parameters. Novel elements have conceptual designs with budgetary pricing and significant qualification items. Integration between systems has moderate design maturity but significant interface uncertainty. Comparison with previous generation systems provides reasonable baseline data, but technological advances create estimating challenges.

4.02 Lithography Systems:

New 1nm lithography technology is still evolving from prototype to production systems. Equipment specifications are preliminary with significant refinement expected. Pricing is based on limited vendor data with large contingency factors. Historical cost trends provide some guidance but technological step-changes limit comparability. This package has the highest uncertainty in both scope and cost.

4.05 Metrology & Process-control Systems:

Advanced metrology systems have moderate design maturity with reasonable vendor budgetary quotes. Integration requirements are still evolving as process parameters are finalised. Software development costs have significant uncertainty due to custom requirements. Testing protocols are at conceptual stage with limited cost data available for validation.