FlipWitness
Coding & Agentic Engineering

A patch needs a witness

FlipWitness

A test is a witness only when it fails before the patch and passes after it.

Nemotron proposes the tests. Before-and-after results supply the evidence.

01 / Input

Choose a case to examine

Three small bugs. Two candidate patches each.
Candidate patch

Paste only code you are allowed to share. Do not include personal data or secrets.

Samples show recorded results for $0. Run live calls the API. Golden tests are hand-written and run locally.

Optional judging access

Loading sample…

Session cost $0

02 / Context

Inspect the report and the change

Bug report

Loading the selected case…

Original code


          
Candidate patch · lines before / after

        

      

      

        
03 / Pipeline
From report to reproducible test

        
NVIDIA Nemotron models via Nebius Token Factory. Sample stages below are recorded results.

        
    
          
  1. Extract the bug report
    Nemotron · Nano
    Expected behavior, actual behavior and steps to reproduce appear here.
    2. 
          
  2. Hypothesize the root cause
    Nemotron · Ultra
    Hypotheses connect the report to the candidate patch.
    3. 
          
  3. Write candidate tests
    Nemotron · Super
    Candidate tests · latest round
    4. 
          
  4. Compare before and after
    Your browser · Web Worker
    Live and golden tests run in your browser. Samples display saved results.
    5. 
          
  5. Rewrite if there is no witness
    Nemotron · Super
    Feed results back into test generation, with at most three rewrites. A witness stops the loop, even if another test flags the patch.
    6. 
        

      

      

        
04 / Evidence card
What do the tests establish?

        
Not run yet

        
Before-and-after results will determine whether a test is a witness.

        

        

          
Each test on the original code and the patched code
TestBeforeAfterEvidence

        

        
A witness supports the patch for that case. It does not prove the patch correct. Errors and timeouts are inconclusive; inspect their messages.