Methodology & Data Analysis Check

Models in Use: Financial Modelling, Agency, and Politics in the Global Financial CrisisResearch title withheld — sample published with the author's permission
International Political Economy · 15 actionable recommendations

Overall assessment

This dissertation presents a theoretical and empirical exploration of financial modeling, moving beyond traditional representationalist accounts by defining models as 'artefacts[extract from the author’s document removed]models in use' framework applied to three distinct institutional cases: Carlyle Capital Corporation, Lehman Brothers, and CalPERS. The work provides a significant contribution by reframing financial fragility not as a failure of calculation, but as a byproduct of continuous financial innovation facilitated by these artefactual instruments. However, the methodological defensibility of these claims relies upon a rigorous disentanglement of strategic agency from external pressures. The study’s primary reliance on legal and bankruptcy-related discovery materials introduces a notable risk of narrative bias, as these documents are inherently structured by litigation objectives. To maintain the robustness of the empirical findings, it is necessary to contrast these discovery accounts with contemporaneous market data and to distinguish between forced accounting adjustments (e.g., FAS157) and proactive strategic modeling. Furthermore, the ‘diverse case’ strategy requires more explicit justification; the shared context of the financial crisis risks masking the study’s broader analytical potential. A more transparent audit trail for the digital archaeology used in the CalPERS case, alongside a clear decision matrix for the tripartite taxonomy, would enhance both the reproducibility and the diagnostic utility of the conceptual framework. Ultimately, the thesis offers a compelling, if complex, argument for the political and narrative nature of modeling in finance. The recommendations provided—ranging from implementing robustness checks for litigation bias to formalizing criteria for model enactment—are intended to strengthen the bridge between the study’s philosophical claims and its empirical execution. By systematically parsing the influence of exogenous market volatility, regulatory mandates, and human administrative discretion, the research can more effectively isolate the ‘constructive agency’ of models as political tools. Addressing these methodological considerations will ensure that the ‘models in use’ framework stands as a coherent and resilient contribution to the literature on critical finance and institutional justification.

What is already defensible

Priority sequence

  1. Perform a 'litigation-bias' robustness check
  2. Clarify 'Diverse Case' selection rationale
  3. Disentangle accounting artifacts from strategic modeling intent
  4. Define criteria for 'performative' vs. 'artefactual' model enactment
  5. Formalize rules for the 'tripartite' framework taxonomy
  6. Acknowledge 'liquidity-induced model failure' as a competitor explanation
  7. Bridge 'material flexibility' and 'human administrative hierarchy'
  8. Account for institutional isomorphism in 'depoliticization' thesis
  9. Micro-macro linkage of 'fictional expectations'
  10. Control for internal 'model divergence' across desks
  11. Control for regulatory divergence as a confounding variable
  12. Establish an audit trail for digital archives
  13. Typology of model 'internal resources'
  14. Caveat archival bias in digital snapshots
  15. Systematize classification of 'manual' vs 'automated' model inputs

Detailed recommendations

Finding 1 · Medium · Strongly Recommended

Clarify 'Diverse Case' selection rationale

data_generation_measurement · Empirical case study analysis

Issue

The selection strategy utilizes CCC, Lehman Brothers, and CalPERS as a 'diverse case strategy[extract from the author’s document removed]models in use' framework beyond crisis-specific environments.

Solution

Provide an explicit justification that isolates how these specific entities capture essential variance across the investment chain. You should clarify whether the 'models in use' framework is intended to be a diagnostic tool for crisis periods or a general theory of model deployment. If the latter, perform a sensitivity check by discussing how the model uses documented in these firms might differ from counterparts that remained stable or experienced different regulatory pressures during the same period, thereby strengthening the structural validity of your chosen cases.

Evidence anchor: “Diverse case strategy”
Finding 2 · Low · Would Strengthen

Establish an audit trail for digital archives

transparency_reproducibility · Empirical case study analysis

Issue

The reconstruction of CalPERS internal institutional materials relies on digital archaeology via the WayBack Machine. Currently, the methodology lacks a formal, itemized log detailing the specific snapshots accessed. This omission creates a transparency deficit, as future researchers or auditors cannot verify the exact state of the digital data at the time of your analysis. Without a clear provenance trail, there is a risk that the selection of snapshots could be perceived as biased toward materials that support the thesis, rather than providing a comprehensive overview of the available historical archives.

Solution

Include a supplementary table or an appendix documenting the specific URL, the exact timestamp of each archived snapshot, and the underlying rationale for selecting that specific version. This documentation serves as a formal audit trail, ensuring reproducibility and demonstrating the rigor of your data gathering process. By transparently mapping your digital evidence, you mitigate concerns regarding snapshot selection bias and allow readers to verify the historical consistency of the materials used in your interpretation of CalPERS' model deployment.

Evidence anchor: “WayBack Machine for digital archaeology”
Finding 3 · High · Required For Defensibility

Perform a 'litigation-bias' robustness check

data_generation_measurement · Empirical case study analysis

Issue

The reliance on primary legal and bankruptcy records, such as the Valukas Report, introduces a systemic narrative bias. These documents are structured specifically for litigation, often focusing on liability-avoidance and the strategic framing of corporate decision-making. There is a tangible risk that your analysis treats these defensive legal narratives—which characterize models as neutral 'tools' or 'problem-solving devices'—as objective business reality. If this framing is accepted without qualification, the study may inadvertently reinforce the very narratives intended to minimize firm responsibility for institutional fragility.

Solution

You must include a reflexive comparative section that contrasts these legal discovery accounts with contemporaneous market commentary, industry trade press, or independent analyst reports from the 2005–2009 period. This triangulation is required to validate your interpretation of 'model as tool' against external perspectives that were not crafted for judicial defense. By grounding your qualitative findings in multiple, independent sources, you can successfully disentangle the strategic legal framing of these models from their actual functional use, thereby ensuring the robustness of your empirical claims.

Evidence anchor: “Legal case files”
Finding 4 · Medium · Strongly Recommended

Disentangle accounting artifacts from strategic modeling intent

integration_interpretation · Empirical case study analysis

Issue

The analysis of Lehman Brothers[extract from the author’s document removed]s autonomy in modeling practices. If the influence of exogenous market volatility on model behavior is not carefully parsed, the conclusion that models served as active instruments for internal strategy may be conflated with mere compliance-driven survival mechanisms.

Solution

Consider explicitly distinguishing between models deployed due to market-imposed necessity and those utilized as a matter of internal strategic preference. You may need to provide evidence from board minutes or internal communications that demonstrate the agency of Lehman’s leadership in choosing specific modeling approaches over available alternatives. By explicitly acknowledging the role of external accounting standards and market illiquidity, you can more precisely calibrate your findings regarding the firm’s strategic use of models during the crisis.

Evidence anchor: “FAS157 implementation”
Finding 5 · Medium · Strongly Recommended

Define criteria for 'performative' vs. 'artefactual' model enactment

integration_interpretation · Conceptual framework development

Issue

The conceptual framework utilizes the 'artefactual' approach to shift focus away from performativity. However, the theoretical distinction between models as passive 'artefacts' and models as 'performative enactments' remains under-defined. Without established boundary conditions, this distinction risks appearing as an analytical preference rather than a empirically grounded category. If readers cannot identify the specific threshold where a tool transitions from a passive resource to a directive, constructivist force, the conceptual framework may lose its diagnostic utility for evaluating institutional finance practices.

Solution

Articulate a set of signposts or a diagnostic matrix that defines the tipping point where a model transitions from an 'artefact'—used for navigation and narrative justification—to a 'performative enactment' that fundamentally directs and creates reality. You should justify this boundary by linking it to your empirical observations of firm behavior, such as when model outputs trigger automated risk-management actions versus when they are used to debate qualitative strategy. Clarifying these operational criteria will strengthen your framework by providing a clear heuristic for other scholars to distinguish between various modes of model deployment.

Evidence anchor: “Many lives of models”
Finding 6 · Medium · Would Strengthen

Account for institutional isomorphism in 'depoliticization' thesis

alternative_explanations · Empirical case study analysis

Issue

The thesis posits that CalPERS adopted specific models as a strategic instrument for depoliticization. However, this interpretation may overlook broader pressures from international bodies like the IMF or OECD, which mandated the adoption of 'modern risk management' standards during the same period. By focusing primarily on internal strategic intent, the study risks conflating proactive institutional masking with normative, external institutional adaptation. Failing to consider these exogenous pressures may weaken the claim that the deployment of these models was a deliberate political maneuver by the board, as the institution may have been fulfilling external compliance expectations rather than exercising autonomous political agency.

Solution

Integrate a discussion that evaluates whether external institutional isomorphism acted as an independent driver for model adoption. The researcher should assess if the 'modern risk management[extract from the author’s document removed]s internal appetite for depoliticization. Justifying this distinction will clarify whether model deployment represents a unique political strategy or a standardized institutional reaction to external pressures.

Evidence anchor: “CalPERS implementation of ‘modern risk management’”
Finding 7 · Low · Would Strengthen

Systematize classification of 'manual' vs 'automated' model inputs

measurement_data_generation · Empirical case study analysis

Issue

The study does not systematically differentiate between model outputs generated through automated historical path simulations and those subjected to discretionary human interventions. This distinction is significant because discretionary inputs often serve a different epistemic and political function than rigid, automated calculations. If these two processes are not clearly separated, the analysis may blur the lines between the model as an 'artefactual' tool and the model as a predictive device. Ignoring this nuance complicates the understanding of how financial actors manipulate data to influence decision-making outcomes, particularly when evaluating how management discretion potentially replaced automated, standardized data inputs within the firm.

Solution

Develop a clearer mapping of the 'degrees of freedom' within the models analyzed. Specifically, the researcher should categorize instances where human intervention or manual adjustment occurred versus those that relied on automated computational processes. Documenting where management override occurred, particularly regarding FAS157 implementation, will provide evidence for the argument that models were actively manipulated rather than serving as passive, programmatic drivers of financial calculation.

Evidence anchor: “FAS157 implementation”
Finding 8 · Medium · Strongly Recommended

Formalize rules for the 'tripartite' framework taxonomy

transparency_reproducibility · Conceptual framework development

Issue

The application of the 'tripartite' framework—comprising position, process, and audiences—lacks explicit classification rules for scenarios where model use spans multiple categories simultaneously. Without established criteria to determine how these overlaps should be resolved, the application of the framework appears potentially subjective. This ambiguity risks undermining the reproducibility of the methodology for future research, as different analysts might classify identical model uses into disparate categories based on personal interpretation rather than a structured rubric. The consequence is a loss of rigor in the conceptual framework, which diminishes its robustness as a tool for comparative analysis across the diverse institutional cases examined in this study.

Solution

Provide a clear decision matrix or a set of operational rules for applying the tripartite taxonomy. The researcher should define the threshold for prioritizing a specific category when a model use overlaps with multiple functions. Establishing these guidelines will ensure consistency in how the conceptual framework is deployed and enable other researchers to reproduce the analytical categorization consistently across different empirical evidence sets.

Evidence anchor: “Tripartite categorization of 'position', 'process', and 'audiences'”
Finding 9 · Medium · Strongly Recommended

Acknowledge 'liquidity-induced model failure' as a competitor explanation

alternative_explanations · Conceptual framework development

Issue

The thesis attributes model failure primarily to inherent internal deficiencies or the political usage of the model by its users. However, this approach may overlook the impact of exogenous liquidity shocks that characterize market collapse. By not explicitly addressing the possibility of 'liquidity-induced failure[extract from the author’s document removed]s performance from broader market dynamics during the Lehman Brothers crisis, the findings regarding the 'political function' of model failure may be interpreted as mere correlation rather than causal influence.

Solution

Formally acknowledge a 'liquidity-induced failure' hypothesis as an alternative perspective. The researcher should contrast this exogenous explanation with their 'constructive agency[extract from the author’s document removed]s influence during extreme market events.

Evidence anchor: “Lehman Brothers Valuation”
Finding 10 · Low · Would Strengthen

Typology of model 'internal resources'

analysis_fit · Conceptual framework development

Issue

The study's definition of 'internal resources' fluctuates between technical code and broader organizational context. This lack of clarity hampers the 'artefactual' claim, as it remains ambiguous whether these resources are immutable, rigid technical features or mutable, discursive components. If these internal resources are not clearly grounded, the argument that models act as narrative templates for fictional expectations becomes difficult to verify. Distinguishing between what is material—such as technical calculation limits—and what is discursive—such as the ability to re-frame assumptions—is critical to understanding how the models functioned as political instruments. Failure to categorize these resources prevents a deeper understanding of how institutional actors leverage different model components to achieve specific organizational or political goals.

Solution

Create a brief typology of 'internal resources' used by the models. The researcher should classify these into distinct categories such as 'data-entry flexibility', 'assumption-masking', and 'reporting-standardization'. Defining these types will provide a firmer basis for distinguishing between the model's technical constraints and its discursive functions, thereby strengthening the empirical application of the 'models in use' framework.

Evidence anchor: “The Key Features of 'Models in Use'”
Finding 11 · Medium · Strongly Recommended

Micro-macro linkage of 'fictional expectations'

analysis_fit · Conceptual framework development

Issue

The current framework characterizes 'fictional expectations' as both psychological heuristics and material 'design fantasies[extract from the author’s document removed]models in use' framework.

Solution

The researcher should explicitly specify whether the 'model-as-narrative[extract from the author’s document removed]Narrating the Liquidity Cushion' anchor.

Evidence anchor: “Narrating the 'Liquidity Cushion'”
Finding 12 · Medium · Strongly Recommended

Control for internal 'model divergence' across desks

robustness · Empirical case study analysis

Issue

The analysis centers on centralized Valuation Control Groups (VCG) while largely omitting the potential existence of shadow models utilized at the trading desk level. This focus creates a potential blind spot regarding the internal heterogeneity of model deployment within the Lehman Brothers case. If trading desks maintained models that conflicted with the official VCG valuations, the study’s implicit framing of models as monolithic entities is undermined. Failing to address this divergence risks overlooking critical evidence that could further illustrate the 'many lives' of models, which is central to the researcher's primary ontological and empirical claims.

Solution

The researcher should acknowledge the potential for model divergence, specifically contrasting desk-level valuation practices with centralized VCG mandates. By explicitly discussing these instances where models were used differently across organizational tiers, the researcher can utilize this variance to strengthen the 'artefactual' claim. This adjustment demonstrates that models are not monolithic, but are instead subject to tactical manipulation by various actors within the firm, thereby reinforcing the central thesis using the existing data from the Lehman Brothers valuation analysis.

Evidence anchor: “Lehman Brothers Valuation”
Finding 13 · Low · Would Strengthen

Caveat archival bias in digital snapshots

transparency · Empirical case study analysis

Issue

The reliance on the WayBack Machine to access historical digital archives is subject to a form of survivorship bias. Because digital repositories often prioritize public-facing, finalized, or sanitized documents, there is a risk that sensitive working drafts or nuanced internal communications were not captured. If the archive predominantly contains information that is intentionally presented for public consumption, the findings regarding 'depoliticization[extract from the author’s document removed]s conclusions remain grounded in a realistic assessment of the available data.

Solution

The researcher should include a brief methodological caveat in the empirical analysis section concerning the potential for selection bias in surviving digital records. This transparency note should acknowledge that public archives, such as those used for the CalPERS study, may favor documents that appear more formal or sanitized compared to internal working files. By explicitly documenting this constraint, the researcher enhances the credibility of the analysis and demonstrates reflexivity regarding the inherent limitations of archival digital archaeology in the context of institutional record-keeping.

Evidence anchor: “CalPERS digital archive”
Finding 14 · Medium · Strongly Recommended

Bridge 'material flexibility' and 'human administrative hierarchy'

analysis_fit · Conceptual framework development

Issue

The current framework attributes substantial agency to the material 'artefact[extract from the author’s document removed]s design and the power dynamics of the administrative hierarchy that dictates its usage.

Solution

The researcher should delineate which aspects of model manipulation are enabled by the material flexibility of the model—such as the inherent 'openness[extract from the author’s document removed]Key Features of Models in Use' section, the researcher can reconcile the artifactual agency with human management. This clarification ensures that the framework accurately reflects how materiality and organizational power co-constitute the deployment of models in the financial crisis.

Evidence anchor: “Key Features of 'Models in Use'”
Finding 15 · Low · Would Strengthen

Control for regulatory divergence as a confounding variable

sampling_and_case_selection · Empirical case study analysis

Issue

Differences observed in model application across the selected cases may be attributable to specific compliance mandates, such as the implementation of FAS157 versus Guernsey law, rather than the intrinsic characteristics of the 'models in use[extract from the author’s document removed]s assertion that models exhibit 'many lives' could be challenged by critics who argue the observed differences are primarily forced outcomes of external compliance pressures.

Solution

The researcher should incorporate a methodological caveat discussing how distinct regulatory environments interact with model usage as a potential confounder. By explicitly comparing how different legal jurisdictions or accounting standards (such as FAS157) impose specific constraints on model implementation, the researcher can better isolate the effects predicted by their 'models in use' framework. This nuance will strengthen the robustness of the diverse case sampling strategy used for the Carlyle Capital Corporation case study.

Evidence anchor: “Carlyle Capital Corporation case study”