An MES demo script that exposes the exceptions
Use a common MES demo script with normal work and revealing exceptions. Separate observed evidence from supplier claims and unresolved assumptions.
Use the same MES demonstration script for every shortlisted supplier and include the exceptions your plant must handle. Record observed behavior separately from slides, future promises and configuration assumptions so the evaluation remains reviewable.
Prepare a compact fictional production story
Create a small set of sanitized items, materials, operations and orders with known expected outcomes. Include only enough detail to test the buying decision. Do not send confidential recipes, customer records or employee data just to make the demonstration look realistic. Tell suppliers the important scenarios in advance so the session tests fit rather than presentation improvisation.
State what may be preconfigured and what the team wants to observe live. Ask suppliers to identify custom code, partner components and roadmap functions. A prepared environment is acceptable when its assumptions are clear. It becomes misleading only when the buyer treats an unimplemented promise as demonstrated production behavior.
Run the normal order first
Have an operator role receive released work, view the intended instruction, identify material, record production and complete the selected operation. Observe which decisions are automated and which require manual action. Confirm the record produced and its destination. Use a supplier demonstrator if necessary, but note where specialist knowledge is needed.
| Observation | What to record |
|---|---|
| Instruction | Version and operation context |
| Material | Acceptance and rejection behavior |
| Quantity | Units, partial completion and validation |
| Quality | Hold, review and disposition path |
| Completion | Record and downstream posting |
| Handover | What the next shift sees |
Do not turn the first smooth order into a final score. It establishes a baseline for the harder cases that follow.
Introduce controlled exceptions
Change an instruction after order release, scan a disallowed material, enter an incorrect quantity and apply a quality hold. Ask the supplier to show the expected user path and resulting records. If a scenario is not configured, record it as not demonstrated and request a bounded follow-up rather than forcing an unsupported improvisation.
Test a duplicate or delayed completion through the proposed interface where feasible. Ask how rejected transactions are detected and resolved. A feature label such as integration or traceability is not evidence of correct exception behavior. Retain the exact scenario and observed result so the team can compare suppliers without relying on memory or presentation style.
Examine roles and operating burden
Include operator, supervisor, quality and support perspectives. Ask what each role must do and what information is available when work stops. Observe whether a correction requires excessive privilege or a technical workaround. Consider the physical workstation and shift environment even if the demonstration runs in a meeting room.
Have observers record facts before discussing overall impressions. A compelling presenter can make unresolved behavior seem less important. Separate usability observations, mandatory controls, configuration questions and commercial exclusions. Do not publish claims about a supplier based on an unverified impression. The evaluation record should remain a buyer’s evidence pack with clear limitations.
Request evidence beyond the screen
Ask for a sample export, an interface reconciliation, a role matrix and an explanation of recovery dependencies. Where a live failure test is not appropriate, request documentation and plan a controlled pilot test. Mark the evidence level: observed, documented, supplier-stated or unresolved. Those categories should not be treated as interchangeable.
Connect gaps to the RFP and proposed statement of work. If a demonstration relies on a module or service not included in the quote, have the supplier reconcile the scope. Ask what the buyer must provide before the same behavior can be achieved at the plant. A demonstration can reveal requirements but does not eliminate implementation work.
Close with follow-up tests and decision gates
Review mandatory criteria first, then compare preferences and cost. Assign unresolved questions to an owner and evidence deadline. A weighted score can help organize preferences only after hard constraints are understood. Do not average away a missing production control because the application has attractive reporting.
If a pilot is needed, define the specific uncertainty it must retire. Carry the demonstration script forward and add plant data, devices, people and recovery conditions. Preserve what was observed and what remains unproven. The final selection should explain why the supplier and scope fit the production problem, what limitations remain and how those limitations will be controlled before rollout.
Take this to the working meeting
Give observers a simple evidence sheet with the scenario, expected result, observed result and follow-up owner. Ask each person to write observations before group discussion. Retain the original sheets and reconcile differences openly. If the supplier explains that a configuration change is needed, record the proposed change and request another test. Do not rewrite an unobserved outcome as a passing demonstration after the meeting.
FAQ
Should we surprise the supplier?
Usually the goal is comparable evidence, not a performance contest. Share scenarios and require transparent assumptions.
Can slides count as proof?
They can explain a design or claim. Keep them distinct from observed configured behavior and test evidence.
How should unsupported scenarios be scored?
Record them explicitly, especially when mandatory. Resolve the gap or reject the fit rather than hiding it inside an average score.
Continue your decision
- MES risk review: test the failures before launch
- MES and OT security: define the trust boundaries
- Electronic records: establish Part 11 applicability
- Lot genealogy: prove the trace in both directions
Use the pilot readiness to document your own inputs. Compare operating models on the comparison page.
Sources and scope
- NIST Smart Manufacturing — Official research context for manufacturing systems. The decision worksheets in this guide are editorial recommendations, not NIST requirements.
Examples are hypothetical and are not measured customer results, supplier quotations, or evidence of professional review. Prepared on October 30, 2026; confirm current requirements with the relevant official body and your responsible specialists.
