Show, Don't Tell Demanding Live Walkthroughs of Your System's Logic

Published on: Sun Oct 20 2024 by Ivar Strand

“Show, Don’t Tell”: Demanding Live Walkthroughs of Your System’s Logic

The standard software demonstration is a familiar exercise. It typically involves a polished slide deck and a carefully scripted tour of the system’s user interface, highlighting its most appealing features. While useful for a general overview, this method is insufficient for rigorous fiduciary due diligence.

A presentation is a claim; it is not evidence. To truly understand a system’s capabilities and limitations, you must move beyond the script and demand a live demonstration of its logic under realistic conditions. You must insist that the vendor or your IT team “show, not just tell.”

The Limitations of the Scripted Demonstration

A scripted demonstration is designed to follow the “happy path”—the ideal scenario where every input is correct, every user acts as expected, and the process flows without friction. By its nature, it avoids the difficult questions:

• How does the system handle non-standard or erroneous data?
• How does it enforce complex, multi-stage approval workflows specific to your grant agreements?
• What happens when it encounters an edge case that falls outside its standard logic?

Relying solely on such a presentation means you are assessing a marketing narrative, not the operational reality of the software. For anyone with fiduciary responsibility, this is an unacceptable basis for decision-making.

A Protocol for a Live Logic Walkthrough

A more robust method of verification is the live logic walkthrough. This is not an adversarial process but a structured inquiry designed to replace assumptions with facts. At Abyrint, we employ a standard protocol for these sessions.

Step 1. Define and Document Your Test Cases in Advance. Before the meeting, identify three to five critical and challenging use cases that reflect the complexities of your actual operations. Do not use generic examples. A good test case might be: “Demonstrate the process for a payment to a sub-grantee that requires co-funding from two different grants with different currency denominations and reporting requirements.”

Step 2. Provide Test Cases with Limited Notice. Share the specific scenarios with the demonstrating team shortly before the session (for instance, 24 to 48 hours). This is sufficient time for them to prepare the necessary data but prevents the development of a new, highly polished script. The goal is an authentic interaction with the system as it currently exists.

Step 3. Control the Navigation. During the walkthrough, either request control of the system yourself or provide explicit, step-by-step instructions to the person navigating. This prevents them from glossing over important details. Be prepared to interrupt the flow to ask clarifying questions: “Stop there. Please show me the configuration screen that governs that specific rule.”

Step 4. Prioritize the “Unhappy Paths.” The most revealing tests are often those designed to fail. Ask the team to demonstrate how the system prevents a user from approving their own expense report or how it flags a potentially fraudulent duplicate invoice from a vendor. The system’s error messages and blocking mechanisms are more indicative of its true control structure than its standard transaction processing.

Step 5. Document the Evidence. Record the session if permitted. At a minimum, take detailed notes and screenshots of key screens, particularly of configuration settings and error messages. This creates a factual record of the system’s demonstrated capabilities, which can be compared against vendor claims and your formal acceptance criteria.

From Demonstration to Due Diligence

The purpose of a live walkthrough is to gather evidence. When a system fails to handle a test case cleanly, it is not necessarily a failing of the product. It is a critical data point that reveals a gap between your requirements and the system’s current state. This knowledge is essential for effective risk management, implementation planning, and ongoing monitoring.

A slide deck is a promise. A live walkthrough is a proof point. For stakeholders who depend on sound governance, only the latter is an acceptable foundation for trust.