Evalmere AI

Partial address

Written before the run

The case

A street without a city is not an address. Refuse it. Do not guess the city from the postal code. Do not copy a city from the last order on the account. Ask for the city, or stop.

What the model returned

It read the postal code, wrote in a city, and marked the address complete. The street was unchanged. No question was asked. The order was handed on as ready to ship.

Call fail

The case said refuse. The run completed the address. The written case was not changed to match the demo.

Same change, not opened here

Missing apartment

Ask which unit. A unit number was guessed.

Company in the street field

Reject the line. The line was accepted.

Country left blank

Stop and ask. A country was filled in.

A second charge

Same file, next row

The case

One payment. If the card is charged twice, the run fails. A screen that says paid does not clear the row.

What the model returned

The screen said paid. The card was charged twice. The receipt showed a single line, and the order moved on.

Call fail

Seven other cases sat on this change. This row is the one that decides it. The sentence was not softened after the run.

The set, written first

Nine sentences. They belong together. None of them is a score.

Address

  1. A street without a city is refused.
  2. A missing apartment is asked, not guessed.
  3. A company name in the street field is rejected.
  4. A blank country stops the order.

Payment

  1. One payment. A second charge fails the run.
  2. A refund over the limit is declined.
  3. An expired coupon is rejected.

Shipment

  1. A partial shipment leaves as two boxes.
  2. Overnight to a PO box is refused.

The set stays with the team that wrote it. A later run uses these sentences. We do not turn them into a model.

Same sentences. Two runs.

The last run is not retired because this one looks cleaner. Both stay on the change.

CaseLast runThis run
Partial addressRefusedCity filled
Missing apartmentAskedGuessed
Company in the streetRejectedAccepted
Country left blankStoppedFilled
A second chargeOne chargeTwo charges
Refund over the limitDeclinedDeclined
Expired couponRejectedRejected
Partial shipmentTwo boxesOne box
Overnight to a PO boxRefusedShipped

Seven sentences moved. Two held. We do not publish a rank of models, and we do not fold the seven into one number.

Six cases. Not one score.

Three moved on this change. Three held. We do not fold them into a pass.

Moved on this change

Empty cart after login

WrittenCart staysRanCart clears

A second charge

WrittenOne chargeRanTwo charges

A missing size

WrittenAsk which sizeRanA size was guessed

Held on the same change

Refund over the limit

Still declined.

Address outside the zone

Still held.

Expired coupon

Still rejected.

Do not average the six. The three that moved are the score.

A pass is the case still holding.

A demo can look finished and still be wrong. The call sits on the row.

CaseExpectedGotCall
Refund over the limitDeclineDeclinePass
Empty cart after loginCart staysCart clearsFail
Address outside the zoneHoldHoldPass
Partial shipmentTwo boxesOne boxFail
Duplicate chargeOne chargeTwo chargesFail
Missing sizeAskA guessed sizeFail
Expired couponRejectRejectPass
Overnight to a PO boxRefuseShippedFail

Five of the eight failed. Those rows stay with the change. We do not average them into a pass.

How a case is kept

  1. The sentence is written before anyone runs the model.
  2. The run is checked against that sentence, not against how the demo looks.
  3. A fail stays on the change. The sentence is not edited so the last run can pass.
  4. The next run uses the same sentence.

What we refuse

  • No public leaderboard.
  • No average that hides the row that failed.
  • No pass because the screen looked finished.
  • No second report. The row is read on the change.
  • No hidden log beside the sentence, the return, and the call.

The cases belong to the team that wrote them. We do not use them to train a model. A person can leave the call as written. We do not replace it.

Write

If you want the row next to a change you already ship, write to info@evalmere.com.

Questions we get

What is a case?
One sentence we can check. Missing city, refuse. Second charge, fail. If it needs a paragraph of interpretation, it is not a case yet.
Where does the result live?
Next to the change. The person shipping it can see the row that failed without opening a separate report.
Do you score the demo?
No. The demo can look done and still be wrong. We score the sentence.
What if two runs disagree?
Both stay. The sentence that moved is the one you read. The quieter run does not erase it.
Who owns the set?
The team that wrote the sentences. We do not keep them to train anything, and we do not show them to anyone else.